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Acronym 
1MB 
3D 
3DIC 
ACE 
AHB 
ADC 
AEC 
AES 
AF 
AFRL 
AMD 
AMS 
ARM 
AXI 
BAE 
BGA 
BOK 
BTMR 
BYU 
CAN 
CBRAM 
ccl 
CGA 
CMOS 


CN 


COTS 
CRC 
CREME 
CREME MC 
CSE 

cu 
D-Cache 
DCU 
DDR 
DFF 
DMA 
DSP 
dSPI 
DTMR 
Dual Ch. 
DUT 
ECC 
EDAC 
EEE 
EMAC 
EMIB 
EPCS 
ESA 
eTimers 
ETW 
FCCU 
FeRAM 
FinFET 
FIR 
FPGA 
FPU 

EXC 

Gb 

Gbps 
GCR 
GEO 
GIC 
GOMACTech 
GPIO 
GPU 


GRC 
GSFC 


Definition 
1 Megabit 
Three Dimensional 
Three Dimensional Integrated Circuits 
Absolute Contacting Encoder 
Advanced high performance bus 
Analog to Digital Converter 
Automotive Electronics Council 
Advanced Encryption Standard 
Air Force 
Air Force Research Laboratory 
Advanced Micro Devices Incorporated 
Agile Mixed Signal 
Acorn Reduced Instruction Set Computer Machine 
Advanced extensible interface 
British Aerospace 
Ball Grid Array 
Body of Knowledge 
Block triple modular redundancy 
Brigham Young University 
Controller Area Network 
Conductive Bridging Random Access Memory 
Correct Coding Initiative 
Column Grid Array 
Complementary Metal Oxide Semiconductor 
Xilinx ceramic flip-chip (CF and CN) packages are ceramic column 
grid array (CCGA) packages 
Commercial Off The Shelf 
Cyclic Redundancy Check 
Cosmic Ray Effects on Micro Electronics 
Cosmic Ray Effects on Micro Electronics Monte Carlo 
Crypto Security Engin 
Control Unit 
defered cache 
Distributed Control Unit 
Double Data Rate (DDR3 = Generation 3; DDR4 = Generation 4) 
Flip-flop 
Direct Memory Access 
Digital Signal Processing 
Dynamic Signal Processing Instrument 
Distributed triple modular redundancy 
Dual Channel 
Device under test 
Error-Correcting Code 
Error detection and correction 
Electrical, Electronic, and Electromechanical 
Equipment Monitor And Control 
Multi-die Interconnect Bridge 
Extended physical coding layer 
European Space Agency 
Event Timers 
Electronics Technology Workshop 
Fluidized Catalytic Cracking Unit 
Ferroelectric Random Access Memory 
Fin Field Effect Transistor 
Finite impulse response filter 
Field Programmable Gate Array 
Floating Point Unit 
Fiscal Year 
Gigabit 
Gigabit per second 
Galactic Cosmic Ray 
geostationary equatorial orbit 
Global Industry Classification 
Government Microcircuit Applications and Critical Technology 
Conference 
General purpose input/output 
Graphics Processing Unit 
NASA Glenn Research Center 
Goddard Space Flight Center 


Acronym 
GSN 
GTH/GTY 
GIMR 
HALT 
HAST 
HBM 
HDIO 
HDR 
HiREV 
HMC 
HOST 
HP Labs 
HPIO 
HPS 
HSTL 
VF 
vO 
120 
i2MOS 
IC 
|-Cache 
JFAC 
JPEG 
JPL 


JTAG 
KB 
L2 Cache 


LCDT 

LEO 

LET 
L-mem 
LANL 

i?) 

LUT 
LVCMOS 
LVDS 
LVTTL 
LTMR 

LW HPS 
M/L BIST 
Mil-STD 
MAPLD. 
MBMA 
MFTF 
uPROM 
uSRAM 
Mil/Aero 
MIPI 
MMC 
MOSFET 
MP. 

MP 
MPFE 
MPSoC 
MPU 
Msg 
MTTF 
NAND. 
NASA 
NASA STMD 
Navy Crane 
NEPP 
NGSP. 
NOR 


Acronyms 


Definition 
Goal Structured Notation 
Transceiver Type 
Global TMR 
Highly Accelerated Life Test 
Highly Accelerated Stress Test 
High Bandwidth Memory 
High Density Digital Input/Output 
High-Dynamic-Range 
High Reliability Virtual Electronics Center 
Hybrid Memory Cube 
Hardware Oriented Security and Trust 
Hewlett-Packard Laboratories 
High Performance Input/Output 
High Pressure Sodium 
High speed transceiver logic 
interface 
input/output 
Inter-Integrated Circuit 
Microsemi second generation of Rad-Hard MOSFET 
Integrated Circuit 
independent cache 
Joint Federated Assurance Center 
Joint Photographic Experts Group 
Jet propulsion laboratory 
Joint Test Action Group (FPGAs use JTAG to provide 
access to their programming debug/emulation functions) 
Kilobyte 


independent caches organized as a hierarchy (L1, L2, etc.) 


NEPP low cost digital tester 

Low Earth Orbit 

Linear energy transfer 

Long-Memory 

Los Alamos National Laboratory 

Low Power 

Look-up table 

Low-voltage Complementary Metal Oxide Semiconductor 
Low-Voltage Differential Signaling 

Low —-voltage transistor-transistor logic 

Local triple modular redundancy 

Lightwatt High Pressure Sodium 
Memory/Logic Built-In Self-Test 

Military standard 

Military Aerospace Programmable Logic Device 
Model-Based Missions Assurance 

Mean fluence to failure 

Micro programmable read-only memory 

Micro SRAM 

Military/Aerospace 

Mobile Industry Processor Interface 
MultiMediaCard 

Metal-Oxide-Semiconductor Field-Effect Transistor 
Microprocessor 

Multiport 

Multiport Front-End 

Multiprocessor System on a chip 
Microprocessor Unit 

message 

Mean time to failure 

Negated AND or NOT AND 

National Aeronautics and Space Administration 
NASA's Space Technology Mission Directorate 
Naval Surface Warfare Center, Crane, Indiana 
NASA Electronic Parts and Packaging 

Next Generation Space Processor 

Not OR logic gate 


Acronym 


NRO 
OCM 

PC 

PCB 
PCle 
PCle Gen2 
Pconfiguration 
Pfunctional_logic 
PHY 

PLL 
PMA 
POR 
Proc. 
PS-GTR 
PSEFI 
Psystem 
QDR 
QFN 
QML 
QsPl 
RADECS 
RC 

R&M 
RAM 
ReRAM 
RGB 

RH 

RT 

SATA 
SCU 

SD 
SD/eMMC 
SD-HC 
SDM 
SEE 

SEFI 

SEL 
SERDES 
SET 

SEU 

Si 

SK Hynix 
SMDs 
SMMU 
SNL 
SOA 
SOc 

SPI 
SSTL 
TBD 
Temp 
THD+N 
TMR 
T-Sensor 
TSMC 
UART 
UltraRAM 
USB 
VNAND 
WDT 
WSR 
XAUI 
XGXS 
XGMII 
XWSG 


Definition 
Naval Research Laboratory 
National Reconnaissance Office 
On-chip RAM 
Personal Computer 
Printed Circuit Board 
Peripheral Component Interconnect Express 
Peripheral Component Interconnect Express Generation 2 
SEU cross-section of configuration 
SEU cross-section of functional logic 
Physical layer 
Phase Locked Loop 
Physical Medium Attachment 
Power on reset 
Processing 
High Speed Bus Interface 
SEU cross-section from single event functional interrupts 
System SEU cross-section 
quad data rate 
Quad Flat Pack No Lead 
Qualified manufactures list 
Serial Quad Input/Output 
IEEE Radiation and its Effects on Components and Systems 
Resistor capacitor 
Reliability and Maintainability 
Random Access Memory 
Resistive Random Access Memory 
Red, Green, and Blue 
Radiation Hardened 
Radiation Tolerant 
Serial Advanced Technology Attachment 
Secondary Control Unit 
Secure Digital 
Secure Digital embedded MultiMediaCard 
Secure Digital High Capacity 
Spatial-Division-Multiplexing 
Single Event Effect 
Single Event Functional Interrupt 
Single event latchup 
Serializer/deserializer 
Single event transient 
Single event upset 
Silicon 
SK Hynix Semiconductor Company 
Selected Item Descriptions 
System Memory Management Unit 
Sandia National Laboratories 
Safe Operating Area 
Systems on a Chip 
Serial Peripheral Interface 
Sub series terminated logic 
To Be Determined 
Temperature 
Total Harmonic Distortion Plus Noise 
Triple Modular Redundancy 
Temperature-Sensor 
Taiwan Semiconductor Manufacturing Company 
Universal Asynchronous Receiver/Transmitter 
Ultra Random Access Memory 
Universal Serial Bus 
Vertical NAND 
Watchdog Timer 
Windowed shift register 
Extended 10 Gigabit Media Independent Interface 
10 Gigabit Ethernet Extended Sublayer 
10 Gigabit Media Independent Interface) 
Xilinx Security Working Group 
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Outline vasa 


¢ FPGA test guidelines 

¢ Microsemi RTG4 heavy-ion results. 

¢ Xilinx Kintex-UltraScale heavy-ion results. 

¢ Xilinx UltraScale+ single event effect 
(SEE) test plans. 

¢ Development of a new methodology for 
characterizing SEU system response. 

¢ NEPP involvement with FPGA security 
and trust. 
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NEPP —- Processors, Systems on a Chip (SOC), ana By 
Field Programmable Gate Arrays (FPGAs) 


State of the Art 
COTS 


“Space” FPGAs 
Sb enn CMOS *Xilinx MPSOC+ 

vege ; -ESA Brave (future) 

Eoreroset | ME Tostee" Pca 

AMD ’ ; (future) 
eT e-Velalcers COTS FPGAs 
Processor Xilinx Kintex+ 
Units (GPUs) *Mitigation 
“Intel, AMD, Nvidia evaluation 
oa =tak-le)i tale me r=) 


“TBD: Microsemi 
processing PolarFire 


leeclatalclarare 

— *Processors: Navy 
= EYelFhitela Best Crane, BAE/NRO- 
la elgel=1al-ve| : -FPGAs: AF, 
Hise Practices Reeve 

ve uation Tate! *Microsemi, Xilinx, 
*BA 2 Synopsis 
-Vorago equi (e (=I fat=s> 
Chilled cekexelalice) |(=16-9) 


£1 O10] 0) Com A\-1 gol) oF-(er=) 


Potential pid task areas: 


artificial intelligence (Al) hardware, Intel Stratix 10 
To be presented by Melanie Berg at the NASA feig arts and Packagin Al) Electronics Technology Workého, ‘ETW), Greenbelt, MD, June26-29, 2017 


FPGA SEU Test Guidelines S 


¢ Impact to community: 

— Currently it is difficult to compare device under test (DUT) test 
data because of differences in test vehicle and test 
methodology 

— The FPGA SEU Test Guidelines Document creates 
standardized test methodologies and provide a means for data 
comparison across organizations and FPGA types. 

— The FPGA SEU Test Guidelines Document points out best 
practices for DUT test structures, monitoring DUT functional 
response, visibility in DUT operation, DUT control, and DUT 
power. 


¢ Update of the test guidelines will be available by 
December 2017. 
— Additional test structures. 
— Embedded processor testing techniques. 
https://nepp.nasa.gov/files/23779/fpga_radiation_test_guidelines_2012. per 
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Various Triple Modular Redundant (TMR) 
Schemes Implemented in FPGA Devices 


= 


Block diagram of block Block diagram of local Block Diagram of 

TMR (BTMR): a complex | TMR (LTMR): only flip- distributed TMR (DTMR): 
function containing flops (DFFs) are the entire design is 
combinatorial logic (CL) | triplicated and data- triplicated except for the 
and flip-flops (DFFs) is paths stay singular; global routes (e.g., clocks); 
triplicated as three voters are brought into voters are brought into the 
black boxes; majority the design and placed design and placed after the 
voters are placed at the jin front of the DFFs. flip-flops (DFFs). DTMR 
outputs of the triplet. masks and corrects most 


single event upsets (SEUs). 


TMR can be embedded in the FPGA or user inserted. 
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FPGA Devices Manufactured as Space- 
Grade Products: Microsemi RTG4 FPGA 


Microsemi’ RTG4” FPGA 


f ] Standard Cell/ 
(> SEU Immune 


FlashBased/ 
SEU Immune 


Up to 24 Lanes, Multi Protocol 3.125 Gbps SERDES 


PMA | PMA PMA PMA 
—_— = . 
. 
- oe XAUI Native SERDES) | 
x1, x2, x XGXS EPCS 


| Upto2 Per Device 


[owe XGMIl, Direct 20-Bit Bus 


System Controller FPGA Fabric Up to 16 SpaceWire Clock & 
150K Logic Elements ‘ ’ Data Recovery Circuits 
POR Generator 
Math Blocks Micro SRAM Large SRAM 
(18x18) (64x18) (1024x18) ‘ , uPROM 
STAG is eee Ps e 

a 462 nee 210 209 

AXVAHB | ~=—S 667 Mbps DDR 
be 
RT PLLs Controller/PHY 
Math Blocks Micro SRAM Large SRAM = : — = 

(18x18) (64x18) (1024x18) AXI/AHB , 667 Mbps DDR 
RC OSC Controller/PHY 


Multi-Standard GPIO 
(1.2 V-3.3 V, LVTTL, LVCMOS, LVDS, HSTL/SSTL, PCl) 
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Impact to Community 
Microsemi RTG4 FPGA 


¢ Next generation to the space- 
grade Microsemi RTAXs. Microsemi ATGA-FPGA 


- |/O interfaces are significantly ae 
more robust versus prior Up to 24 Lanes, Multi Protocol 3.125 Gbps SERDES 
Microsemi space-grade FPGAs ee PP spe 
devices. oe 

¢ Embedded mitigation, cauiedead 1s Loge Semen ES 
packaging, and qualification — kek er 
process makes this device ——— eee 
space-grade. = a oe i 06 

¢ Flash based configuration — eae 


hence configuration is 
essentially SEU immune. 


¢ Embedded flip-flop (DFF) SEU 
hardening. 

NEPP performs an independent study to determine the level 
of SEU susceptibility for the various RTG4 components. 
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Microsemi RTG4: Device Under Test (DUT) vasa 
Details 


¢ New Entry into the Aerospace Market with Space-grade 
Expectation. 


— Bulk UMC 65nm CMOS process with an epitaxial layer. Flash based 


configuration. 


— Qualified to MIL-STD-883 Class B, and Microsemi will seek QML Class 


Q and Class V qualification. 


- The DUT : RT4G150-CG1657M. peli ep ba eres 


SRAM: sequential random access memory. 


¢ We tested Rev B and Rev C devices. 2$?-: digital signal processing. 
¢ The DUT contains; 


PLL: phase locked loop. 


158214 look up tables (4-input LUTs); 

158214 flip-flops (DFFs); 720 user I/O; 

210K Micro-SRAM (uSRAM) bits; 

209 18Kblocks of Large-SRAM (LSRAM); 

462 Math logic blocks (DSP Blocks); 

8 PLLs; and 48 global routes (radiation-hardened global routes); 
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Microsemi RTG4: Device Under Test (DUT) vasa 
Embedded Hardening 


Sglerteneet Gate DFFs are radiation hardened 
ecesed i“ using LTMR and SET filters 
Routing placed at the “f input. 

l Vertical 
| a Routing 
_ RTG4 SEU - Protected 
oatin Control = .- 
Floating ra ! Flip-Flop 


Hardened configuration flash cell 
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Microsemi RTG4 Study Objectives S 


¢ This is an independent investigation that evaluates the single 
event destructive and transient susceptibility of the Microsemi 
RTG4 device. 

¢ Design/Device susceptibility is determined by monitoring the 
DUT for Single Event Transient (SET) and Single Event Upset 
(SEU) induced faults by exposing the DUT to a heavy ion 
beam. 

¢ Potential Single Event Latch-up (SEL) is checked throughout 
heavy-ion testing by monitoring device current. 

¢ The objectives of this study are the following: 

— Analyze flip-flop (DFF) behavior in simple designs such as shift 
registers. Compare SEU behavior to more complex designs such 


as counters and finite impulse response (FIR) filters. Evaluating 
data trends helps in extrapolating test data to actual designs. 


— Analyze global route behavior — clocks, resets. 
— Analyze configuration susceptibility. 
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DUT Preparation 


NEPP has populated two Rev B and four 
populated Rev C boards with RT4G150- 
CG1657M devices. 

The parts (DUTs) were thinned using 
mechanical etching via an Ultra Tec 
ASAP-1 device preparation system. 

The parts have been successfully thinned 
to 70um — 90um. 


0193} Le 


. ia as Be i e 1 
eee 1-1-4 
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Challenges for Testing sal 


TID: total ionizing dose 


¢ Software is new... place and route is not optimal yet. Hence, it 
is difficult to get high speed without manual placement. 

¢ Microsemi reports that devices show TID tolerance up to 
160Krads. 


— Although, when testing with heavy-ions, dose tolerance will be 
much higher. 
— TID limits the amount of testing per device. 
— Number of devices are expensive and are limited for radiation 
testing. 
— A large number of tests are required. 
- We will always need more parts. 
¢ Current consortium participants: 
— NEPP (Goddard and JPL), 
— Aerospace Corporation, and 
— Microsemi. 
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Test Setup 


Labview GUI Connected to Memory 
Processing in HSDT. Commands are 
also sent (and echoed) to the H SDT 
through this RS232 interface 


Logic Analyzer Connected 
Labview GUI connected to —_ to WSR or Counter Outputs 


WSR or Counter Processing 


as) 


80.00 


CLK SRA 


Rb232(1) 
7X232(1) 


DUT INPUTS 
DUT/Outputs 


CLK 
RESET 


Processing 
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Microsemi RTG4 Designs Tested S 


Test Structure a a=xe [U(-vaveamat-lale[= 


Global routes 2KHz — 150MHz 
shift Registers (WSRs) 2KHz — 150MHz 
Counters 5MHz — 100MHz 


Finite impulse response filters 1MHz-100MHz 
(FIRS). Math-block (DSP) 
testing 


Embedded SRAM N/A 


Test structures selected in order to investigate specific 
RTG4 components and data trends across a variety of 
designs. 
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Windowed Shift Registers (WSRs): 
Test Structure 


N levels of Inverters ; —« DFF = D flip flop 
between DFF stages: - 4-bit Window Output 
N = 0, 8, and 18 


Shift Register Chain 


p= © DaeO oY = ©) Di@ ip) =" ©) Die Dee) (oy = ©) 


bp" @Q 


OI 


> 


SS ly wsr —_—_——> 
WS Ry D Q | D Q 
wo) 2G 


WSR, 
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Counter Arrays 


DUT contains two sets of the 
following: 


— 200 8-bit counters 
— 200 8-bit snapshot registers 


All counters and snapshot registers 


are connected to the same clock 
tree and RESET. 

The clock tree is fed by the CLK 
input from the LCDT. 

DUT CLK is connected to a DGBIO 
and a CLKBUF. 

The LCDT sends a clock anda 
reset to the DUT. The controls are 
set by the user 


Counter 
Processing 


ferme 


Low Cost 
Digital 
Tester 


Shift Up 
Registers 
Every 4 
Clock 
cycles 


a 
i 


ounters can still operate after most SEUs. However after an 
occurs, the tester must recalculate a new expected value for the 
affected counter. 


2 sets of counter 
arrays are tested 
simultaneously 
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Test Facility Conditions: Texas A&M vasa 
University Cyclotron Facility 
¢ 25 MeV/amu tune. 


¢ Flux: 1 x 104 to 5 x 10° particles/cm?'s 


¢ Fluence: All tests were run to 1 x 10’ particles/cm? or 
until destructive or functional events occurred. 


¢ Test temperature: Room temperature 


MEV/Nucleon 0° 60 ° 
He 25 .07 .14 


N 25 4) 18 
Ne 25 1.8 3.6 
Ar 25 5.5 11.0 
Kr 25 19.8 40.0 
Xe*™ 25 38.9 78.8 


**We were unable to obtain Xe during our testing 
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Test Facility Conditions: Lawrence Berkel 
National Laboratory 
» 16MeV/amu tune. CYClotron Facility 


¢ Flux: 1 x 104 to 5 x 10° particles/cm?'s 


¢ Fluence: All tests were run to 1 x 10’ particles/cm? or until 
destructive or functional events occurred. 


¢ Test temperature: Room temperature 


‘lon... | Energy (MEV/Nucleon) | LET (MeV*cm2/mq) 0° 


N 16 1.16 
Ne 16 2.39 
Si 16 4.35 
Ar 16 7.27 
V 16 10.9 
Cu 16 16.5 
Kr 16 25 


Xe 16 49.3 
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Heavy-lon Configuration Re- vasa 
programmability Results 


¢ During this test campaign, tests were only 
performed up to an LET of 49.3MeVcm2/mg. 

¢ Higher LETs will be used during future testing. 

¢ No re-programmability failures were observed up 
to an LET of 49.3MeVcm2/mg when within particle 
dose limits. We did not try to reprogram while the 
beam was turned on. 
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Heavy-ion Global Route Results Nasal 


¢ Global routes are the backbone of all designs. Hence, it is 
imperative to investigate global route SEU susceptibility. 


¢ For NEPP DUT test structures, clock trees were connected to a 
variety of global clock tree sources: 

— Direct clock I/O 

— Internal Oscillator 

— Clock conditioning circuit (PLL) 

— TMR clock conditioning circuit (TMR PLL) 

¢ Summary of global route results starting from best performance: 

— Direct clock I/O had the lowest SEU susceptibility (best performance. 

— Clock conditioning circuit had higher SEU susceptibility than direct 
clock I/O. However, performance can still be acceptable for critical 
missions. 

— TMR clock conditioning circuit (TMR PLL) did not appear to reduce 
susceptibility and might have higher susceptibility at higher 
frequencies. 

— Internal OSC is SEU soft and should not be used in critical circuits. 


To be presented by Melanie Berg at the NASA Electronics Parts and Packaging (NEPP) Electronics Technology Workshop (ETW), Greenbelt, MD, June26-29, 2017 21 


Rev C: WSRs with SET FILTER versus vasa 
LET at 100MHz 


Ape 0a" Add combinatorial logic, 
__ pe increase cross section. 
aaa How and what dated x @ WSR16 Checkerboard 
seo _ thee : big = WSR8 Checkerboard 
_ : 4 WSR4 Checkerboard 
u. iene x WSRO Checkerboard 
< x WSR16 All 1's 
S mane Bae - @WSR8 All 1's 
 : ; = WSR4 All 1's 
© -WSRO All 1's 
2.00E-09 - x -WSRI6 All 0's 
#WSR8 All 0's 
1.00E-09 x fa =WSRG4 All 0's 
rs 4WSRO All 0's 
0.00E+00 7 a <n 
0 5 10 15 20 25 
LET MeV*cm?2/mg 
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WSR and Counter Accelerated Radiation vasa 


Test Data Observations 

WSR chains showed a variety of dependencies (all are 
as expected): 

— Increase clock frequency- increase failures. 

— Increase combinatorial logic — increase failures. 

— Increase data change rate — increase failures. 
Use of flip-flop SET filter — decrease failures. 
Use of flip-flop SET filter - decreases system operation speed. 
As LET increases, the effectiveness of the SET filter 
decreases. This is because generated SETs become wider 
(more energy) and have more power to defeat the SET filter. 
Results (SET filter on) are in-line with the Microsemi 
SEU radiation hardened predecessor — Microsemi 
RTAXs family. 


However, the Microsemi RTAXs family had slightly 
better SEU performance. 
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RTAX4000D and RTAX2000 WSRs at 
S8OMHz with Checkerboard Pattern 


1.00E-06 , RTAX4000D and RTAX2000 have better SEU 
performance than RTG4 (higher LET,,,,52:; and 
4.00E-07 slightly lower Oseus); 
but not by much. 
_ 1.00E-08 
LL. 
LL. 
> 1.00E-09 
E ; 
2 aS ~~ RTAX4000D WSR, 
ob — RTAX4000D WSR, 
1.00E-11 -— RTAX2000 WSR, 
-— RTAX2000 WSR, 
1.00E-12 — 
0 20 40 60 80 
LET (MeV*cm?/mg) 
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SEU Cross-Section (cm2/bit) 


Microsemi SRAM SEU Cross-Section @y 
versus LET 


_ EE 


1.00E-06 - Total Bit Upsets 
BMultibit Upsets 
1.00E-07 - @ - Singlebit Upsets 
a * - ; 
~ X Total Bit Upsets Static 
1.00E-08 ne @Multibit Upset Static 
~ @Singlebit Upsets Static 
1.00E-09 - - a 1.00E-06 
Fd x x : 
1.00E-10 - @ 
a e& 1.00E-07 
1.00E-11 i 
1.00E-12 eal 
= 
— 
1.00E-13 - —, § 1.00E-09 
0 20 40 60 3 
2 ”) 
LET MeVcm2/mg ‘ 1.00E-10 
2 
oO 
Large SRAM G 1.00E-11 - 
Microsemi included 
a ‘ 1.00E-12 - 
SRAM bit interleaving. 
Makes a difference with 1.00E-13 


EDAC. 


EDAC was only tested with 


pSRAM 
ySRAM 


@MicroMem Total bit Upsets 

& MicroMem Multibit Upsets 
MicroMem SingleBit Upsets 

X MicroMem EDAC Total Bit Upsets 
MicroMem EDAC Multibit Upsets 

@ MicroMem EDAC Singlebit Upsets 

&: MicroMem TotalBit Upsets Static 

-MicroMem Multibit Upsets Static 
MicroMem Singlebit Upsets Static 


10 


20 30 40 50 60 
LET MeVcm2/mg se 
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Microsemi Math Block Test Structures @® 


P_=P,,.,+CARRYIN+C+/-(A0*BO) 


SUB 
ae — 18x18 multiply 
accumulate math-blocks. 
jovertow — Dual redundant chains 


BO[17:0] 


[43:0] with a compare. 


C[43:0] T 2Ewe 
— Coefficients are shared. 


CARRYIN 


ARSHFT 
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RTG4 Math-block (DSP) SEU Cross ay 


1.00E-06 


SEU Cross-Section (cm?/DSP) 


1.00E-07 - 


1.00E-08 - 


1.00E-09 - 


1.00E-10 - 


1.00E-11 - 


1.00E-12 - 


0 


Sections versus LET 


DSP No Feedback As LET increases, the 

= DSP No Feeback SET SET filter is not as 

ADSP Feedback . o.. 
effective. This is as 


x DSP Feedback SET 
expected. ¢ 


- 
x 
A 
x 18x18 
N a b d DSP multiply 
o errors opserve 
No Feedback SET filter accumulate 
math-blocks 
1 2 3 4 5 6 


LET MeV*cm?2/mg 
2 
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Deliverables: Microsemi RTG4 Test 


Report Submission and Data Summary 


¢ Two versions of reports have been submitted. 
¢ Third version will be completed by June 2017. 
- Asasummary: 
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RTG4 is not as SEU hardened as it’s predecessor (RTAXs). 
However, fairly close. 

Exception: embedded SRAM with EDAC - is better in the RTG4. 
Embedded TMR PLL does not operate as expected. No 
improvement in SEU susceptibility. 

Internal Oscillator PLL is highly susceptible to SEUs. 

Designs implemented in the RTG4 device do not operate as fast as 
they do when implemented in the RTG4’s predecessor (RTAXs). 
This is most likely due to the place and route software. However, 
this is unexpected. 

TBD for NEPP to perform more testing. At this point, additional 


testing is assumed to be funded by partners or missions. 
28 


SRAM-based FPGA Mitigation Study o 
using Xilinx Kintex-Ultrascale 
(XCKU040-1LFFVA1156l) 
(1) Single event latch-up (SEL) 
and (2) Mitigation 


Impact to Community vasa 
Kintex-UltraScale 


Osey: SEU Cross-section IP: intellectual property 
Next generation of FPGA devices from the commercial Xilinx-7 
series. 
I/O interfaces are significantly more robust 
There are no embedded mitigation. However, additional gate- 
count better allows the user to insert mitigation into the 
design. 
There is no embedded processor. However, the user can 
embed a soft-core or use the Zync-UltraScale (contains hard-IP 
processor cores). 


P (fs ) system LF Configuration 1 P (fs ) functionalLogic + Foepr 


Design Ogcey Configuration Ogey Functional logic SEFI Osey 
OsEU 
NEPP performs an independent study to determine the level 


of SEU susceptibility for the various FPGA components. 
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Xilinx Kintex-Ultrascale S 


¢ New Entry into the Aerospace Market with COTS 
Expectation ... 20 nm planar process (TSMC). 


Data Transfer Is Key for Our New System Applications: 
UltraScale Transceivers 


ne  @Tal te ereliig-texer-1 (= Virtex UltraScale 


Type GTH GTY GTH GTY 
Quantity 16-64 0-32 20-60 0-60 


Maximum Data 16.3Gb/s 16.3Gb/s 16.3Gb/s 30.5Gb/s 
Rate 


Minimum Data 0.5Gb/s 0.5Gb/s 0.5Gb/s 0.5Gb/s 
Rate 


Key Applications Backplane Backplane Backplane 100G+Optics 
PCle PCle PCle Chip-to-Chip 
Gen4 Gen4 Gen4 25G+ Backplane 
HMC HMC HMC HMC 
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Xilinx Kintex-UltraScale Study vasa 
Objectives 


¢ This is an independent investigation that evaluates the single 
event destructive and transient susceptibility of the the Xilinx 
Kintex-UltraScale device. 

¢ Design/Device susceptibility is determined by monitoring the 
DUT for Single Event Transient (SET) and Single Event Upset 
(SEU) induced faults by exposing the DUT to a heavy ion 
beam. 

¢ Potential Single Event Latch-up (SEL) is checked throughout 
heavy-ion testing by monitoring device current. 

¢ This device does not have embedded mitigation. Hence, user 
implemented mitigation is investigated using Synopsys 
mitigation tools. 

¢ FPGA part# XCKU040-1LFFVA1156l. 


¢ Collaboration: Xilinx and Synopsys. 
Beam time was limited: SEL, configuration, and 


Mitigation. 
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TMR Description al 


DFF: Edge triggered flip-flop; L: Combinatorial Logic 


TMR DY =x-Yor gl e}iteyal TMR 
Nomenclature Acronym 


Block TMR Entire design is triplicated. Voters are BIMR 
placed at the outputs. 

Local TMR Only the DFFs are triplicated. Voters LTMR 
are placed after the DFFs. 

Distributed TMR DFFs and CL-data-paths are DTMR 


triplicated. Similar to a design being 
triplicated but voters are placed after 


the DFFs. 
Global TMR DFFs, CL-data-paths and global GTMR or 
routes are triplicated. Voters are XTMR 


placed after the DFFs. 


Note: It has been suggested to separate (partition) TMR domains in 
SRAM based designs so that there are no overlapped shared 
resources. Shared resources become single points of failure. 
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DTMR Par LITiIONIN g . i 


SEUs that occur in one 
TMR domain are 
expected to be 
mitigated. 
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Kintex-Ultrascale Designs Tested S 


Test Structure acre LU(=laleaYm at: lale[= 


Counter Array No TMR 5OMHz 
Counter Array DTMR with 50MHz 
partitioning 

Counter Array DTIMR no 5OMHz 
partitioning 

Counter Array BTMR with 5OMHz 
partition 

Counter Array LTMR with 5O0MHz 
partition 


NEPP has the only current heavy-ion data for the Synopsys 
mitigation tool. 
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Test Facility Conditions sal 


¢ Facility: Texas A&M University Cyclotron Single 
Event Effects Test Facility, 25 MeV/amu tune). 


¢ Flux: 1 x 104 to 5 x 10° particles/cm?'s 


¢ Fluence: All tests were run to 1 x 10’ particles/cm? or 
until destructive or functional events occurred. 


Test temperature: Room temperature 


oo oF a 


N 25 9 18 
Ne 25 1.8 3.6 
Ar 25 5.5 11.0 
Kr 25 19.8 40.0 
Xe** 25 38.9 78.8 


We were unable to obtain Xe during our testing 
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Xilinx Scaling Family Trends for 
Configuration Bits in Heavy lons 


David Lee et. al. “Single-Event Characterization of the 20 nm Xilinx Kintex UltraScale 
Field-Programmable Gate Array under Heavy Ion Irradiation” 
10" . . https://www.osti.gov/scitech/servlets/purl/1263983 


=k 
o 
J 
1 


UltaScale 
Kintex- 7 
WVirtex-4 
‘irbex-I 


— 
©, 


Cross-Section (cm?/bit) 


| a 4 1 | I i i 
oO 10 20 30 40 50 60 fo 


LET (MeV*cm?/mg) 
Daily upsets in configuration in LEO and GEO are expected. 
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History of Xilinx and Single Event 
Latchup (SEL) or Latchup-Like Events: 
Virtex 2 through UltraScale Series 


¢ Xilinx Virtex 2: Latchup-like events have been observed in 
flight. Most likely due to embedded half-latches in the 
device. 

¢ Xilinx Virtex 5: Half-latches were removed. No latchup- 
like events observed during SEE testing or in flight. 

¢ Xilinx 7-series: Is it SEL or latchup-like? Observed only 
on 7-series devices that contained 3.3V I/O. Devices that 
do not contain such I/O have no latchup-like events. 


¢ Xilinx UltraScale series no latchup-like event observed. 
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Kintex-UltraScale Mitigation Study: Counter Sy 
Arrays Mean Fluence to Failure (MFTF) versus 


1.00E+08 t LET No TMR 
First observed DTMR — wgBTMr Partition 
m Partition failure 4 DTMR Partition 
_ 100E+07 - = DTMR no Partition 
“ A e@LTMR 
= a a 
—  1.00E+06 - 
g : . 
L é 
2 
= 1.00E+05 - M4 
= 
= 1.00E+04 - . 
LL. = 
= LTMR was not tested at this —7 
1.00E+03 - - | LET , | | 
0 1 2 3 4 5 6 


LET MeV*cm?2/mg 
Kintex-UltraScale Data drops off quicker than radiation hardened 
Xilinx Virtex (V5QV). 


More SEU testing should be performed for more detailed comparisons. 
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Comparison of V5QV and Kintex 
UltraScale with Mitigation 


V5QV Counters Kintex UltraScale DTMR 
1.00E+08 5 Unt 
7 Couniers 
= E A 
= 1.00E+07 - : S 1.06407 - 
< ° a 
cS) °o 7 
© = 
= 1.00E+06 @  1.0E+06 - ‘ 
= "” 
LL e _ 
n 1.00E+05 = 1.0E+05 
@V5QV Counter Filter Off AKintex UltraScale Partition 
mwV5QV Counter Filter ON @Kintex UltraScale No Partition 
1.00E+04 =-— —Ir 71 1.0E+04 —_—_- T ——__i 
0 2 4 6 0 2 4 6 
LET MeVcm2/mg LET MeV*cm?/mg 
Synopsys results are looking good. 
At 
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Summary of Mitigation Application to Kintex- 
UltraScale during SEU-Heavy-lon Testing 


SEFI: single event functional interrupt 
¢ Mitigation study proves DTMR is the strongest mitigation scheme 
implemented in an SRAM-based FPGA. 
— However, for flushable designs BTMR might be acceptable. 
— LTMR is not acceptable in SRAM-based FPGAs for any design. 
— Partitioning may not be necessary. 
¢ Although GTMR has been implemented in V5 families and earlier 
Xilinx device families, NEPP has suggested to avoid GTMR because 
clock skew is difficult to control. 
— In 2015-2016, via heavy-ion SEU testing, It has been observed in the 
Xilinx 7-series, that race conditions due to clock skew are unavoidable. 
— This is due to the speed of combinatorial logic and route delays in the 7- 
series versus earlier Xilinx FPGA device families. 
¢ Synopsis tool has improved for simple designs. They are still 
working on IP core instantiations and other challenges. 
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Deliverables: Xilinx Kintex-UltraScale 
Test Report Submission and Data 
Summary Test Report 
¢ The full Kintex-UltraScale SEU dataset is still 


currently being analyzed and will be available by 
June 2017. 


- Asasummary: 

— NEPP has provided insight into Xilinx potential latchup- 
like events. 

— Through previous testing and design experience, NEPP 
has provided Synopsys with information for sufficient 
mitigation strategies per FPGA. 

— TBD for NEPP to perform more testing. At this point, 
additional testing is assumed to be funded by partners 
or missions. 
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Xilinx Zynq UltraScale+ S 


¢ New Entry into the Aerospace Market with COTS 
Expectation. 


— 16nm FinFet vertical process (TSMC). 


— Depending on mission requirements, additional mitigation 
may be required. 


- Zync UltraScale+ Includes: 


— Dual and quad core variants of the ARM Cortex-A53 (APU). 
— Dual-core ARM Cortex-R5 (RPU). 


— Dedicated ARM graphics processing unit (GPU). 


SRAM based Configuration. No radiation hardening is 
applied to flip-flops. However, manufacturer hopes FinFET 
technology will reduce SEU susceptibility. 
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Xilinx Kintex-Ultrascale+ Transceivers 


Data Transfer Is Key for Our New System Applications: 
UltraScale+ Transceivers 


aS Kintex-Ultrascale+ Nl edeye) Om O] lig: betor-1(-5 


Type GTH GTY PS-GTR = GTH GTY 
Quantity 20-60 0-60 4 0-44 0-28 


Maximum 16.3Gb/s  32.75Gb/s 6.0Gb/s 16.3Gb/s 32.75Gb/s 
Data Rate 


Minimum 0.5Gb/s 0.5Gb/s 1.25Gb/s 0.5Gb/s 0.5Gb/s 
Data Rate 
Key Backplane 100G+Optics PCle Backplane 100G+Optics 
Applications PCle Chip-to-Chip Gen2 PCle Chip-to-Chip 
Gen4 25G+ USB Gen4 25G+ 
HMC Backplane Ethernet HMC Backplane 


HMC HMC 
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Xilinx UltraScale+ Schedule S 


¢ We plan to test two platforms: 
— MPSoC evaluation board. 
— Custom Kintex-Ultrascale+ daughter card. Designed by NEPP. 


¢ We currently have one evaluation board. MPSoC evaluation 
boards (ready for testing) will be in hand in June 2017. 


¢ Proton testing using the MPSoC evaluation board is planned for 
August 2017 timeframe. 


¢ Custom board is planned to be completed October 2017. 
¢ Heavy ion testing will occur FY17 and FY18. 
¢ Current Partners: 
— NASA Goddard Science Data Processing Branch, 
JPL, and 
— Sandia National Laboratory 
— Xilinx 
We are looking for additional collaboration. 


To be presented by Melanie Berg at the NASA Electronics Parts and Packaging (NEPP) Electronics Technology Workshop (ETW), Greenbelt, MD, June26-29, 2017 46 


Development of New Methodology for 
Characterizing SEU System Response (1) 


¢ This study transforms proven classical reliability models into the 
SEU particle fluence domain. The intent is to better characterize 
SEU responses for complex systems. 
¢ Will be discussed in further detail in another ETW presentation. 
¢ Deliverables: 
— Development of analysis (ongoing). 
— Guidelines manual delivery in FY18. 


1.0E+08 }  V5QV: MicroBlaze with Cache @ 1 
Enabled £ 1.0E+02 - 
< 1.0E+07 @ = BV5: PowerPC 0.9999 =" SQV: MFTF= 7.0x10° 
= rs © 1.0E+00 - — PowerPC: MFTF = 2.8x10? 
G 1.0E06- 8 4 r 0,998 
© E 1.0E-02 » 
3 1.0E+05 7 7 0.9997 - We fall below 99.99% at 
2 ° g 1.0E-04 - approximately 
u 1.0E+04 - = 09996 - _0.02particles/cm?! R(®)=¢ 070° 
E ¢ & 1,0E-06 - iene 
LL & R(@)=¢%25s10 
= 1.0E+03 8. E 0.9995 
i i 1.0E-08 
1.0E+02 + : © To 0.07 0.14. 1.8 To 3.6 To 20 To 40 arc (9904 mn 
0 20 40 60 80 100 0.07 To To18 36 20 40 over 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 
LET MeVcm2/mg ms LET Bins (MeVcm?/mg) Fluence (particle/cm’) 
Mean fluence to failure (MFTF) Environment data Reliability Curve i 
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Development of New Methodology for 
Characterizing SEU System Response (2) 
¢ The proposed method does not rely on data-fitting and hence 
removes a Significant source of error. 


¢ The proposed method provides information for highly SEU- 
susceptible scenarios; hence enabling a better choice of mitigation 
strategy. 


¢ This methodology expresses SEU behavior and response in terms 
that missions understand via classical reliability metrics. 


¢ Presentations: 


¢ Government Microcircuit Applications and Critical Technology 
Conference (GOMACTech) 2017 in Reno, NV. 

¢ Single Event Effects (SEE)/Military Aerospace Programmable Logic 
Devices (MAPLD) 2017 in San Diego, CA. 

¢ Submission to IEEE Radiation and its Effects on Components and 
Systems (RADECS) 2017. Conference will be held in Geneva, SUI. 
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FPGA Security and Trust S 


¢ Goal: Support the U.S. government concerns regarding security 
and trust in FPGAs. 


¢ Conference participation: 

— Xilinx Security Working Group (XSWG) 2016 in Longmont, CO. 

— Government Microcircuit Applications and Critical Technology 
Conference (GOMACTech) 2017 in Reno, NV. 

— Hardened Electronics and Radiation Technology (HEART) 2017 
in Denver, CO. 

— Hardware-Oriented Security and Trust (HOST) 2017, McLean, 
VA. 

— Joint Federated Assurance Center (JFAC) FPGA working 
group: Trusted Microelectronics Special Topic: Field 
Programmable Gate Array Assurance Workshop, McLean, VA. 

¢ Collaboration with Aerospace Corporation, JFAC, and other 
agencies. 

— Meetings, consultations, and presentations. 
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Questions? 
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